TeXDYNA: Hierarchical Reinforcement Learning in Factored MDPs
نویسندگان
چکیده
Reinforcement learning is one of the main adaptive mechanisms that is both well documented in animal behaviour and giving rise to computational studies in animats and robots. In this paper, we present TeXDYNA, an algorithm designed to solve large reinforcement learning problems with unknown structure by integrating hierarchical abstraction techniques of Hierarchical Reinforcement Learning and factorization techniques of Factored Reinforcement Learning. We validate our approach on the LIGHT BOX problem.
منابع مشابه
Automated Discovery of Options in Factored Reinforcement Learning
Factored Reinforcement Learning (FRL) is a method to solve Factored Markov Decision Processes when the structure of the transition and reward functions of the problem must be learned. In this paper, we present TeXDYNA, an algorithm that combines the abstraction techniques of Semi-Markov Decision Processes to perform the automatic hierarchical decomposition of the problem with an FRL method. The...
متن کاملIntelligents et de Robotique Hierarchical & Factored Reinforcement Learning
This thesis is accomplished in the context of the industrial simulation domain that addresses the problems of modelling of human behavior in military training and civil security simulations. The aim of this work is to solve large stochastic and sequential decision making problems in the Markov Decision Process (MDP) framework using Reinforcement Learning methods for learning and planning under ...
متن کاملDiscovering Hierarchy in Reinforcement Learning with HEXQ
An open problem in reinforcement learning is discovering hierarchical structure. HEXQ, an algorithm which automatically attempts to decompose and solve a model-free factored MDP hierarchically is described. By searching for aliased Markov sub-space regions based on the state variables the algorithm uses temporal and state abstraction to construct a hierarchy of interlinked smaller MDPs.
متن کاملEfficient Structure Learning in Factored-State MDPs
We consider the problem of reinforcement learning in factored-state MDPs in the setting in which learning is conducted in one long trial with no resets allowed. We show how to extend existing efficient algorithms that learn the conditional probability tables of dynamic Bayesian networks (DBNs) given their structure to the case in which DBN structure is not known in advance. Our method learns th...
متن کاملChi-square Tests Driven Method for Learning the Structure of Factored MDPs
sdyna is a general framework designed to address large stochastic reinforcement learning (rl) problems. Unlike previous model-based methods in Factored mdps (fmdps), it incrementally learns the structure of a rl problem using supervised learning techniques. spiti is an instantiation of sdyna that uses decision trees as factored representations. First, we show that, in structured rl problems, sp...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2010